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Abstract. Online teaching environment 

is a challenge for science, technology, 
engineering, and mathematics (STEM) 
in-service teachers who feel worried about 
their ability to succeed in what might be 

an unfamiliar learning environment. This 
research focused on testing the effects of 
perceived risks and concerns by in-service 
teachers during online-only classes due 

to the coronavirus outbreak on student 
academic progress requirements across the 
different STEM disciplines. The research hy- 
potheses were tested on a sample of 1444 
in-service teachers teaching exclusively 

in the online environment, in Romania. A 
structural equation model was used to ex- 
plore the possible links among two external 
variables (pedagogical and technological 
perceived risks), one mediator variable 
(student engagement), one control variable 
(school settings), and one output variable 
(student academic progress requirements). 
The results revealed significant negative 
paths from challenges mediated by student 
engagement to student academic progress, 
as well as positive paths from them to me- 
diator factor. The school setting categories 
were negatively correlated with both per- 
ceived risk dimensions. The moderator role 
of student engagement on the challenges- 
outcome link was supported. The online 
teaching effects on student’s academic 
progress varied across the different STEM 
disciplines. The relevant common features 
for all STEM disciplines were further then 
demonstrated. 
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Introduction 


The integration of digital learning resources has become the only pos- 
sible way to support authorities to continue educating students during the 
COVID-19 emergency (Reimers et al. 2020)), who profoundly disturbed the 
global economy (Dhawan, 2020). A large research related to impacts of e- 
learning-only strategies used for teaching on student academic performance 
was performed in Romania, between 16 March 2020 and 16 June 2020 (Bot- 
nariuc et al., 2020). This particular educational context has determined the 
researchers to investigate the predictors of student academic progress, from 
the perspective of enhancing the learning outcomes. The exclusive use of only 
online educational strategies predictably increases the doubts on student's 
completed core requirements for each STEM discipline (Wladis et al., 2014). It 
was recognized that adoption of online education does not certainly connect 
to successful curriculum results, mainly concerns are related to the insufficient 
time for planning online activities, the lower quality of student interaction 
and the difficulty in managing online interactions (Porter et al., 2016). 

STEM education focuses on the blend of skills and knowledge acquired 
in each STEM discipline (i.e., mathematics, technologies, informatics, biology, 
chemistry, physics, geography) into a science and technology education- 
oriented teaching (BuyUkdede & Tanel, 2019). Researchers suggested that 
teachers find it difficult to approach STEM subjects interdisciplinarily (Kelley 
& Knowles, 2016, p. 1). Therefore, there is uncertain knowledge regarding the 
factors influencing online course outcomes, common for all STEM disciplines 
(Hachey et al., 2015). This gap in empirical understanding is delaying efficiency 
in virtual teaching for STEM education. The results of present research are 
expected to provide relevant particularities for future teaching STEM subjects 
in online mode. In-service teachers who teach various disciplines believed that 
students educational, pedagogical, technological, and psychological chal- 
lenges and concerns are increasing during only online education (Xu & Jag- 
gars, 2014). This research explored that key idea, in case of STEM disciplines. 
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Conceptual Framework 


This research has focused on teaching challenges met by in-service STEM teachers in online-only activities. 
The previous research studies on these topics (Eltanahy et al., 2020; Giamellaro & Siegel, 2018; Temitope et al., 
2018) suggested that in-service teachers’ perceived risks (pedagogical risks, technological risks, and those caused 
by student’s behavioral problems, labeled emotional engagement) during online class may have effects on stu- 
dents’ interest to fulfill general requirements across each STEM discipline, which is indispensable for the successful 
acquisition of specific skills at the end of the study year. 


Pedagogical Risks 


The most prominent pedagogical risks of online teaching and learning identified in the specialized lit- 
erature include the following: the changes occurred in teacher’s role, the lack of face-to-face interaction and 
a higher time consumption in planning and conducting classes, as well as guiding students (Bettinger et al., 
2017; Xu & Jaggars, 2014; Zhang et al., 2004). The role of teachers is to present the content knowledge into an 
effective mode for learning in accordance with the adequate curriculum ideology (Mnguni, 2019). As a result 
of commuting to online-only classes, educators currently face the need to readjust their teaching strategies, 
methods, forms of organization and, consequently, their traditional role altogether. Teaching and learning can 
no longer be subsumed to a magistro-centric paradigm, where the instructor knows and controls everything 
in his class (Cucos, 2016). The teacher now becomes a knowledge facilitator, a monitor and a guide in students’ 
use of technology (Panisoara et al., 2017), and a mentor who is always adapted to his/ her students’ needs and 
requirements (Darling-Hammond et al., 2020). When this role change does not occur, online teaching and learn- 
ing will be mostly ineffective (Melrose et al., 2013). 

Face-to-face interaction is likewise considered imperative for granting a successful teaching and learning 
process by a large number of educational institutions (Westwood, 2016). The fact that teachers cannot observe 
and assess their students’ immediate reaction to their statements might determine instructors to be indifferent 
or inattentive to their tone or word choice (Brookfield & Preskill, 2012). Concurrently, when they are not able to 
observe students’ reactions in time, teachers might lose their primary tool for self-regulation and adjustment to 
students’ opinions and needs. 

The third main pedagogical risk of teaching online is the lack of a proper time management, together with 
the use of inadequate assessment tools (Hansen, 2019), both leading to a higher time-consumption (Karchmer- 
Klein & Fisher, 2020) than the one needed in face-to-face instruction. This is even more likely to happen in the case 
of asynchronous instruction, where the teaching-learning-evaluation process does not necessarily imply as many 
predetermined time limits as synchronous or face-to-face instruction, and where teachers are required to provide 
guidance and feedback regardless of their usual working time (Stachowiak et al., 2020). 

Other risks may concern the difficulty of a theoretical transformation of education within the new online 
teaching environment, the changes brought to the previously used teaching practices (which also imply teachers’ 
openness to acquire new pedagogical skills), as well as the lack in the ability to adapt and integrate into a new 
learning environment (Hewett & Bourelle, 2018). 


Technological Risks 


A precondition for switching for traditional to online classes is the existence of a proper integration of digital 
learning resources in educational activities. For example, the computer equipment absence becomes one of the 
main technological risks in teaching online. If there are students who do not afford or do not yet possess such an 
equipment, a significant part of school’s target groups will be deprived of an adequate educational experience 
(Gibson & Martinez, 2013). Simultaneously, even when students own the necessary technological tools, they might 
encounter a series of technical difficulties, which will lead to their frustration (Bender, 2003) and a possible loss of 
interest towards online learning. Furthermore, when students have no other alternative or choose to use mobile 
phones instead of a laptop/ desktop computer, learning activities in STEM subjects, such as programming, are 
difficult to execute on a small screen (Kaul, 2020). 

Apart from the above-mentioned difficulties, there is also a series of risks which is placed at the border of 
pedagogy and technology, such as: inefficient use of interactive teaching tools, teacher’s lack of comfort and flex- 
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ibility with using technology or insufficient preparation for providing technical support in an online environment 
(Meloncon, 2007). 


Student Engagement 


Student engagement has been recurrently associated with successful task completion as well as persever- 
ance and contentment towards learning (Ashwin & McVitty, 2015; Reeve et al., 2020; Trowler, 2010). In addition 
to the abovementioned, there are multiple elements which may be indicative of students’ engagement, such as 
the amount of time and energy students dedicate to learning (Alqurashi, 2020), intrinsic motivation and active 
participation in the learning process, self-regulated learning or fulfilment of learning assignments on schedule 
(Bangert-Drowns & Pyke, 2001; Wang & Kang, 2006). In opposition, the lack of the prior, in conjunction with class 
disruption and learning dissatisfaction (Fredricks, 2014), point to students’ disengagement towards the learning 
process. In Kahu’s opinion (2013), engagement epitomizes student’s psychological involvement on three specific 
levels: behavioral, cognitive and emotional. Behavioral engagement refers to student's observable intention to 
actively participate in the learning process, cognitive engagement relates to student's effort to enhance his/her 
thinking processes, whereas emotional engagement is exhibited through student's positive reactions or notice- 
able states, such as enthusiasm, enjoyment etc. (Reeve & Shin, 2020; Skinner, Kindermann, Connell et al., 2009; 
Skinner, Kindermann, & Furrer, 2009). Consequently, engagement refers to the degree in which students involve in 
a learning activity and can be ascertained behaviorally, emotionally or cognitively (Panisoara et al., 2019), through 
students’ observable conducts, attitudes, or reactions. 

When considering the multiple benefits of ICT use, such as permanent access to learning opportunities, 
interactive educational tools, ease of student collaboration or the chance to overcome geographical boundaries, 
one may tend to assume that students should display a high level of involvement in online educational activities. 
Nonetheless, there are only a few studies that approach the differences between online and traditional learning 
in terms of students’ engagement (Boboc, 2018). The use of technology is proved to stimulate students’ focus 
throughout a task and to be adaptable to students’ different needs and learning styles (Farmer, 2003); however, 
as regards students’ emotional engagement, the pedagogical variable is equally important in assuring its occur- 
rence. When students and teachers find it difficult to manage or to employ the educational potential of online 
classes, they may develop frustration or discouragement (Sofka et al., 2012). In this framework, the use of what is 
currently referred to as “cybergogy” — a set of strategies for effective online teaching and learning (Wang & Kang, 
2006) —- becomes imperative for constructing a dynamic and effective learning environment and for increasing 
students’ engagement in the learning process. 


Student Academic Progress 


Academic progress is associated with knowledge acquisition, skill achievement and task accomplishment 
(Reeve et al., 2020) and conventionally occurs when students complete their academic requirements within a 
preestablished time frame and when their final product meets a set of predetermined standards (Scott-Clayton 
& Schudde, 2016). Academic progress has been shown to relate with constructs such as personality traits and 
educational identity dimensions (Klimstra et al., 2012), classroom heterogeneity of students (Kuzmina & lvanova, 
2018), as well as motivation and encouragement (Yusop & Correia, 2018). Lubbers et al. (2006) study showed that 
academic progress is also associated with positive relationships between classmates, as the latter ones improve 
student engagement which, successively, influences student academic progress. 

As regards academic progress, which is expected to occur in an online environment, students reached bet- 
ter educational results when they participated in e-learning (Jung & Huh, 2019; Zhang et al., 2004). However, if 
the switch from traditional to online classes is considered a change in the educational process, then this change 
may as well fall under supported hypothesis, according to which school shifts are associated with an increase in 
behavioral problems Sullivan et al., 2010. Additionally, assuming that traditional learning may sometimes constrain 
teachers to comply to a fixed syllabus without adapting to students’ preferences or learning styles, online learning 
might as well amplify this issue (Lumy & Sajimon, 2019). Consequently, the quality of online courses should be a 
prevalent pedagogical objective (Reevy & Bursten, 2015; Xu & Jaggars, 2014) in order to grant students a differenti- 
ated successful instruction. 
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Research Problem 


The effects related to the student academic progress met by STEM in-service teachers during exclusive online 
teaching was the main research problem. Thus, the technological, pedagogical perceived risks and student's en- 
gagement during exclusively online activities were the possible causes that led to no fulfilment of the curriculum 
accordingly with the initial scheduling at the beginning of the school year. In this context, it was justified to try 
understanding the predictors of the student's academic goal progress for each STEM discipline. For these reasons, 
two external variables that can have an impact on the achievement of common core standards have been identi- 
fied and explored, respectively: pedagogical and technological perceived risks. In this research, the engagement 
by students in online learning activities was considered a mediator variable, and the school settings (i.e., rural, 
medium-urban, and large-urban) as control variable. In-depth knowledge of these five factors of research model and 
the correlations between them will lead to improve STEM online education in the middle school and high school. 


Research Aim and Research Questions 


There is a lot of research about teachers’ perspectives during technology-enhance teaching and learning. 
Several studies point out that teachers have a negative attitude towards online technology-mediated instruction 
(Hennessy et al., 2005; Islahi & Muslim, 2019). According to Brown (2016), teachers manifested technological anxiety, 
complexity, and illiteracy. The lack of instructional design may possibly negatively influence teacher's motivation 
to use digital resources accurately (Alalwan et al., 2020). 

However, no research has been found that teachers perceived technological, pedagogical, and behavioral 
risks during online-only classes influenced by student's completed core requirements in case of STEM disciplines 
taught, which confirmed the need of this research. The main research aim was to understand the challenges and 
concerns of the teachers related to online education, and to analyze the effects of perceived risks by them on stu- 
dent's fulfilled basic educational standards across each STEM discipline (mathematics, technologies, informatics, 
biology, chemistry, physics and geography). 

The research questions were the following: 

Q,. What is the effect of in-service teachers’ pedagogical perceived risks during only online classes on 
student's academic progress for each of the seven STEM discipline? 

Q,. What is the effect of in-service teachers’ technological perceived risks during only online classes on 
student's academic progress for each of the seven STEM discipline? 

Q,. What is the effect of student engagement during online-only classes on their academic progress for 
each of the seven STEM discipline? 

Q,. What is the direct and indirect role of school settings on the academic progress for each of the seven 
STEM discipline? 


Research Methodology 
General Background 


This research used quantitative methods such as structural equation modelling (SEM) (Heyder, 2019) for testing 
causal models that could explain the correlations between observable and latent dimensions (Akilli & Geng, 2017). 
The powerful SEM consists of generating measurement models and structural models to estimate the correlations 
between research model dimensions. The target population of this research was in-service teachers who teach 
online-only STEM disciplines during COVID-19 emergency in Romania between April and June 2020. 


Questionnaire 
The questionnaire was split into two sections. The first part requested socio-demographic data including: 
gender, age, school setting size, number of years of teaching experience, and discipline taught. School setting 


was used as a control variable, and it was indicated the size dimension of the school’s teachers and students. This 
dimension was constructed at three levels of settings: rural, medium-urban, and large-urban. 
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A self-developed tool designed to measure the impacts on students’ learning outcomes of online teaching 
mode across STEM teachers in Romania was used in the second part of the questionnaire. It consisted of 13 items 
and was covered of four scales: pedagogical, with 3 items, technological, 4 items, student engagement, 3 items, 
and academic progress, 3 items. The present research instrument was part of the tool constructed at the beginning 
of coronavirus outbreak for a large cross-sectorial research by experts in educational science domain from Romania 
(Botnariuc et al., 2020). Important tendencies, questions, and gaps that have been summarized in research variables 
were identified going through the literature review (Botnariuc et al., 2020). 

Teachers’ perceived risks regarding pedagogical challenges met during online-only classes were measured 
with 3 self-developed items. Exactly, teachers were requested to rate pedagogical difficulties in performing online 
teaching activities for each discipline due to the following aspects: lack of appropriate tools for online learning- 
evaluation, lack of class management tools, feedback and remote evaluation tools, and lack of pedagogical support 
for achieving sufficiently effective and/or attractive learning activities for all students. 

Teachers were requested to rate technological difficulties in performing online teaching activities for each 
discipline due to the following aspects: insufficient level of digital competence, technical difficulties (complicated 
connection on certain platforms, access restrictions, browser limitations, additional software installations, etc.), 
limited Internet access, and missing a computer/ tablet/ smartphone. Each item of pedagogical and technologi- 
cal dimensions was assessed on a 4-point scale (1 = Not at all, 2 =To a small extent, 3 = To a moderate extent, 4 = 
To a great extent). 

Teachers were requested to rate the engagement level of students during online-only classes respectively: my 
students do not follow the rules, my students are not paying attention in class, and my students do not participate 
actively in class. All items related to student engagement dimension were formulated on a 4-point Likert-type 
response scale (1 = Do not know/ Not applicable, 2 = Disagree, 3 = Partial Agree, 4 = Agree). 

Teachers were requested to rate the academic progress of different type of students during online-only 
classes, respectively: students with different results (i.e., good and very good, medium, bad) succeed to go through 
the Common Core curriculum according to previous plans. Responses to the items corresponding to academic 
progress dimension were coded on a 4-point scale (1 = Not at all, 2=To a small extent, 3 = Sufficient to permit the 
continuation of activities and the “on-going” recovery, 4 = Totally). 


Participants and Ethical Considerations 


A large number of in-service teachers were requested to fill in a questionnaire as volunteers, their beliefs 
related to technological and pedagogical challenges, student engagement and academic progress during online- 
only educational activities due to the coronavirus outbreak, in the spring semester of the 2019-2020 academic 
year. The research has a cross-sectional design, and was performed nationwide in Romania (Botnariuc et al., 2020). 
The sampling criterion for present research was in-service teachers, who are teaching one of the STEM disciplines, 
as follows: mathematics, informatics, technologies, biology, chemistry, physics, and geography. The research par- 
ticipants consisted of 1444 STEM teachers and represented a part of the larger group of respondents (Botnariuc et 
al., 2020). The participant number was suitable for the research purpose and multivariate analysis (Celik & Yesilyurt, 
2013). In present research, the participants’ demographic characteristics can be seen in Table 1. 


Table 1 
Demographic Profile of Voluntary Research Participants 


Discipline (percentage) 


Variables Dimensions mathematics technologies informatics biology chemistry physics geography 
male 17.66 21.02 19.20 6.79 2.94 16.56 28.38 
Gender 
female 82.34 78.98 80.80 93.21 97.06 83.44 71.62 
Rural school practice 29.10 12.10 14.40 32.10 21.32 28.03 31.76 
ne Medium school practice 26.62 36.62 28.00 30.86 34.56 24.20 22.30 
Large school practice 44.28 51.27 57.60 37.04 44.12 47.77 45.95 
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Discipline (percentage) 
less than 5 years 8.46 8.92 7.20 741 0.74 1.27 7.43 
between 6 and 10 years 5.22 4.78 3.20 741 4.41 1.91 7.43 
Seniority between 11 and 20 years 20.90 26.43 18.40 35.19 18.38 7.01 39.19 
between 21 and 30 years 35.82 38.54 56.80 38.89 38.97 49.04 38.51 
more than 30 years 29.60 21.34 14.40 11.11 37.50 40.76 7.43 


All participants were invited to join on research voluntarily. The informed consent, as an agreement for the 
involvement of participants was discussed between all engaged persons in research. So, the respondents were well 
informed about the research, the possible hazards, and the advantages of their contribution. Those who decided 
to join in the research completed an online questionnaire. The research was conducted from April to May 2020 
and has the approval of Ethics Committee of Bucharest University from Romania. 


Research Model and Research Hypotheses 


The research model (Figure 1) was involved and two latent factors as external dimensions, one latent factor as 
a mediator dimension, and student's academic progress varies across STEM disciplines as an outcome dimension. In 
Figure 1, itcan be observed each latent variable involved in the theoretical model: pedagogical risks, technological 
risks, concerns about student's behavioral engagement and student's academic progress for each STEM subject. 
This correlational research used the Structural Equation Modelling (SEM) to explore the structural links between 
STEM teachers perceived risks and student's academic progress for each STEM discipline. 
In this context, based on theory, the hypotheses (Figure 1) settled to test the effect of STEM teachers’ peda- 
gogical and technological perceived risks, and their perceptions regarding students’ engagement on student's 
academic progress during only online classes and their relation to each other, for each STEM discipline were: 
Direct effect 
H.: The technological perceived risks by teachers are positively and significantly correlated with the peda- 
gogical perceived risks by them. 

H.: The pedagogical perceived risks by teachers are positively and significantly correlated with the school 
settings. 

H.: The technological perceived risks by teachers are positively and significantly correlated with the school 
settings. 

H,: The technological perceived risks by teachers positively and significantly affect student engagement. 

H,: The pedagogical perceived risks by teachers positively and significantly affect student engagement. 

H_.: The technological perceived risks by teachers negatively and significantly affect student academic 
progress. 

H_: The pedagogical perceived risks by teachers negatively and significantly affect student academic pro- 
gress. 

H.: The student engagement during only online classes negatively and significantly affects student aca- 
demic progress. 

H,: The school settings positively and significantly affect student academic progress. 

Moreover, the hypotheses established to test the effect of teachers’ pedagogical and technological perceived 
risks, and their perceptions regarding students’ engagement on student's academic progress during online-only 
classes and their relation to each other, for each STEM discipline were: 

H_,,: The positive relationship between pedagogical and technological perceived risks was similar for all 
students who studied STEM disciplines. 

Multigroup effect 

H_,,. The positive relationship between pedagogical perceived risks and school settings was similar for all 
students who studied STEM disciplines. 

H_,. The positive relationship between technological perceived risks and school settings was similar for all 
students who studied STEM disciplines. 
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H_,. The positive relationship between technological perceived risks and student engagement was similar 
for all students who studied STEM disciplines. 

H_,. The positive relationship between pedagogical perceived risks and student engagement was similar 
for all students who studied STEM disciplines. 

H_.. The negative relationship between technological perceived risks and student academic progress was 
similar for all students who studied STEM disciplines. 

H_.. The negative relationship between pedagogical perceived risks and student academic progress was 
similar for all students who studied STEM disciplines. 

H__. The negative relationship between student engagement and student academic progress was similar 
for all students who studied STEM disciplines. 

H_.. The positive relationship between school settings and student academic progress was similar for all 
students who studied STEM disciplines. 


Figure 1 
The Research Model and the Hypotheses 
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Data Analysis 


The factor structures of the dimensions were analyzed using Exploratory Factor Analysis (EFA) conducted in 
IBM SPSS Statistics V22.0 software in order to certify that the measurement models had achieved model fit (Heyder, 
2019). EFA outputs were explored using structural equation model (SEM) (Kline, 2011), supported by IBM SPSS 
Amos version free trial software. SEM was selected due to their advantages of simultaneously estimating links 
between latent and outcome factors (Thibaut et al., 2018). A mixture of exploratory and factor analysis was used 
to test the measurement model (CFA model) validity (Lazar et al., 2020). Multigroup confirmatory factory analysis 
(MGCFA) was used to test equivalence of measures (Milfont & Fischer, 2010) across STEM disciplines. The data set 
was divided into groups (e.g., mathematics, technologies, informatics, biology, chemistry, physics, and geography). 
The model fit was determined for each group separately, and then multi-group comparisons were performed. The 
measurement invariance using MGCFA was recognized if at least the configural invariance, and metric invariance 
were both supported (Steinmetz et al., 2009). Data analysis was conducted going through three recognized stages: 
(1) measurement models which revealed the links between observed and latent variables and (2) structural models 
which highlighted direct and indirect influences of variables in the research model (Celik & Yesilyurt, 2013; Lu et 
al., 2016), and (3) multi-group comparisons to explore whether respondents from different groups understood 
the same construct in a similar way across different STEM disciplines (Lee, 2018). The key objective for stage 1 was 
to evaluate the reliability and construct validity for the measurement model. Composite reliability (CR) with the 
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minimum threshold of 0.7, average variance extracted (AVE) with the minimum threshold of 0.5, maximum shared 
variance (MSV), and McDonald construct reliability (MaxR(H)) were calculated for each model construct to test the 
level of reliability, convergent and discriminant validity. The (Fornell and Larcker’s (1981) criteria for discriminant 
validity were fulfilled if the values of MSVs were less than the values of AVEs the discriminant validity (I. Lazar et 
al., 2020; |. M. Lazar et al., 2020; Panisoara et al., 2020). In the second stage, each hypothesis was tested using path 
analysis (Akilli & Geng, 2017). 


Research Results 
Measurement Model 


The measurement model was developed based on previous research studies and tested if the data fit the 
research model. If model fit criteria were adequate, the measurement model could be accepted. The evaluation 
of the model goodness of fit were considered based on the following criteria: the chi-square/degrees of freedom 
x2/df value is lower than 2, the Goodness-of-Fit Index (GFI) value is higher than .95, the Root mean square error of 
approximation (RMSEA) value being between .00 and 0.06, the Standardized Root Mean Square Residual (SRMR) 
value being between .00 and .08, and the p of Close Fit (PClose) (Celik & Yesilyurt, 2013; Lu et al., 2016). All of these 
values corresponding to measurement model respected the recommended threshold values, as shown in Table 2. 
So, the dataset of all four dimensions was observed to be proper for path analyses (Akilli & Geng, 2017). 


Table 2 
Structural Basic Model Fit Measures 


Measure Estimate Threshold Interpretation 
y-ldf 1.403 Between 1 and 3 Excellent 
CFI 999 >,95 Excellent 
SRMR 019 <.08 Excellent 
RMSEA OTT <.06 Excellent 
PClose 999 >.05 Excellent 


The internal consistency reliability for the four measurement scales on the experimental data used for this 
research was computed with Cronbach’s Alpha, and this reliability coefficient was found .714. The factor loads of 
items, as an outcome of EFA were found to be between .730 and .924. The CFA illustration is presented in Figure 2. 
The impact of online teaching challenges on students’academic progress has four latent variables and 13 observed 
variables. Technological challenges (8=.542, t=15.296) and pedagogical challenges (8=.506, t=12.065) were the most 
crucial latent variables of measurement model. Overall, the measurement model exhibited appropriate convergent 
validity. As shown in Table 3, the Fornell and Larcker’s (1981) criteria for discriminant validity were fulfilled because 
the square root of AVE for each construct was higher than the inter-construct correlations (Lu et al., 2016, p. 50). 


Table 3 
Convergent and Discriminant Validity of Measurement Model 


Category CR AVE MSV MaxR(H) Pedagogical — Technological al eee 
Pedagogical 817 600 § .278 846 115 - 363*** 
Technological 854 596 278 869 S20 ‘te -.362*** 
StudentEngagement .791 560  .101 813 tle 10" 148 -.318"" 
Academic Progress 812 592 135 .826 170 


Note: ***p < .001 
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Figure 2 
Confirmatory Factor Analysis Model 
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The proposed hypotheses were tested using SEM model (Heyder, 2019) with teachers’ perceived risks and 
concerns predicting students’academic progress requirements. The theoretical structural model (Figure 3a) identi- 
fied the impacts of technological perceived risks, pedagogical perceived risks, and student engagement on student 
academic progress for baseline model (Figure 3a), and for each STEM discipline (Figure 3b — 3h). Moreover, SEM 
path analysis method significantly revealed the importance of school settings as covariable. The variable “school 
settings” was introduced in the model with three levels: rural, medium-urban, and large-urban. Causal links from 
school settings to each latent variable in the model were also tested. 

Goodness-of-fit indices (Malik et al., 2018) proved that the structural basic model fits the data excellently 
(x2/df = 1.775, CFl =.999, SRMR =.084, RMSEA =.023, and PClose =.715). Measurement invariance (configural and 
metric) across STEM discipline was reached. Configural invariance was achieved because confirmatory measure- 
ment models proved the reliability of measurement properties, and the metric invariance of measurement was 
validated because x2 of 36.028 with 36 degrees of freedom was not statistically significant a=.467 (Malik et al., 2018). 

The explanatory power of path analyses was evaluated by computation of effect sizes (r). Lenhard and Len- 
hard (2016) quoted from Cohen (1988) that the minimum r should be 0.10. Akilli and Geng (2017) also quoted 
from Hair et al. (1995) that “significant paths showing hypothesized direction empirically support the purposed causal 
relationship”. Appendix A shows the path unstandardized coefficients and their associated t values and p values. 
The outcomes of the baseline model indicated that 24% of the total variance is in academic progress (Figure 3a). 
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Figure 3 
Results of structural equation modelling using full information maximum likelihood 
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Path Analyses Differences Across STEM Disciplines 


The percent of the total variance in academic progress varied across the STEM discipline, between 14% for 
informatics and 36% for geography (Figure 3d and 3h). Appendix A indicates the unstandardized coefficients, t- 
values and p-values in path analyses corresponding to each STEM discipline. The results of the structural model in 
case of mathematics (Figure 3b) indicated that 27% of the total variance in academic progress was explained by 
the model, and the impacts of school settings on academic progress and the correlations between pedagogical 
challenges and school settings were not significant. 

The results of the structural model in case of technologies (Figure 3c) indicated that 26% of the total variance 
in academic progress was explained by the model, and the impacts of school settings on academic progress and 
the correlations between both challenges and school settings were not significant. The results of the structural 
model in case of informatics (Figure 3d) indicated that only 14% of the total variance in academic progress was 
explained by the model, and the impacts of pedagogical, technological challenges and school settings on academic 
progress, and pedagogical risks on student engagement were not significant. The results of the structural model 
in the case of biology (Figure 3e) indicated that 26% of the total variance in academic progress was explained by 
the model, and the impacts of pedagogical and school settings on academic progress, and pedagogical risks on 
student engagement, and also the correlations between pedagogical challenges and school settings were not 
significant. The results of the structural model in the case of chemistry (Figure 3f) indicated that 30% of the total 
variance in academic progress was explained by the model, and the impacts of pedagogical challenges on student 
engagement, technological challenges on academic progress and the correlations between both pedagogical 
and technological challenges and school settings were not significant. The results of the structural model in case 
of physics (Figure 3g) indicated that 18% of the total variance in academic progress was explained by the model, 
and the impacts of school settings and pedagogical challenges on academic progress were not significant. The 
results of the structural model in case of geography (Figure 3h) indicated that 38% of the total variance in aca- 
demic progress was explained by the model, and the impacts of school settings and technological challenges on 
academic progress and the correlations between pedagogical challenges and school settings were not significant. 

Appendix B shows the standardized coefficients, and the effect size (r) values corresponding to significant 
path across all STEM disciplines, respectively: technological perceived risks predicted student engagement, and 
student engagement influences student academic progress. All significant effects (Appendix B - values in bold) 
can be considered small to intermediate in size (Lenhard & Lenhard, 2016). 

MGCFA was used to compare the constrained model versus unconstrained model for each of the nine path 
coefficients corresponding to nine research hypotheses. The results of the MGCFA stated the differences in Chi-square 
value (Ay2) between the constrained model and unconstrained model, associated with the degree of freedom (df) 
and the level of significance p of model comparison (Malik et al., 2018). 

Thus, research model path was validated in the context of online education, when controlled for school settings. 
The causal links and implicitly the validation of each hypothesis differed across STEM disciplines. Accordingly, only 
some research hypotheses H.-H, were validated (Table 4). The outcomes of the multigroup analysis are presented in 
Table 5. For the link between perceived risk of technological issues, respectively perceived risk of pedagogical issues 
and academic progress, the x2 difference was significant (Table 5) suggesting there are differences in the magnitude 
of each corresponding hypothesized path across STEM disciplines. Consequently, H,, and H,, were not confirmed. 


Table 4 
Results of Path Analyses Corresponding to Hypotheses H.-H, 


Hypothesized STEM discipline 
path 
Mathematics Technologies Informatics Biology Chemistry Physics Geography 


H, — Technological<+Pedagogical Confirmed Confirmed Confirmed Confirmed Confirmed Confirmed Confirmed 
; Not con- Not con- Not con- Not con- Not con- 
H, | Pedagogical<+School Settings Fare fened Confirmed feed ae Confirmed Gand 
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Hypothesized 
path 


Mathematics Technologies Informatics 


H,  Technological<+School Settings Confirmed 
H, Bates Engage- Gésnaned 
H, ial Engage- Confirmed 
H, ees Sannaned 
H, | Pedagogical—-Academic Progress Confirmed 
H, es eer Confirmed 
School Settings—Academic Not con- 
° Progress firmed 
Table 5 


Not confirmed 


Confirmed 


Confirmed 


Confirmed 


Confirmed 


Confirmed 


Not con- 
firmed 


STEM discipline 


Confirmed 


Confirmed 


Not con- 
firmed 


Not con- 
firmed 


Not con- 
firmed 


Confirmed 


Not con- 
firmed 


Results of Multigroup Analysis Corresponding to Hypotheses H, -H., 


Hypothesized 
path 
H,,  Technological<+Pedagogical 
H,,  Pedagogical<+School Settings 
H,,  Technological<+School Settings 
H,,  Technological—Student Engagement 
H,, | Pedagogical—-Student Engagement 
H,,  Technological—Academic Progress 
H,, Pedagogical—Academic Progress 
H,, Student Engagement—-Academic Progress 
H,, School Settings—Academic Progress 
Discussion 


Model 


Structural 
covariances 


Structural 
covariances 


Structural 
covariances 


Structural 
weights 


Structural 
weights 


Structural 
weights 


Structural 
weights 


Structural 
weights 


Structural 
weights 


df 


Biology 


Confirmed 


Confirmed 


Not con- 
firmed 


Confirmed 


Not con- 
firmed 


Confirmed 


Not con- 
firmed 


x2 


2.075 


9.652 


11.051 


1.829 


3.638 


14.078 


13.4 


2.427 


7.215 


Chemistry 


Not con- 
firmed 


Confirmed 


Not con- 
firmed 


Not con- 
firmed 


Confirmed 


Confirmed 


Confirmed 


p 


0.913 


0.14 


0.087 


0.935 


0.726 


0.029 


0.037 


0.876 


0.301 


Physics 


Confirmed 


Confirmed 


Confirmed 


Confirmed 


Not con- 
firmed 


Confirmed 


Not con- 
firmed 


THE EFFECTS OF ONLINE TEACHING ON STUDENTS’ ACADEMIC PROGRESS IN STEM 
(PP. 


1106-1124) 


Geography 


Confirmed 


Confirmed 


Confirmed 


Not con- 
firmed 


Confirmed 


Confirmed 


Not con- 
firmed 


Testing Results 


Confirmed 


Confirmed 


Confirmed 


Confirmed 


Confirmed 


Not confirmed 


Not confirmed 


Confirmed 


Confirmed 


The interests in students’academic progress requirements remained key task to monitor the development of 


students’ competencies (Chamandy & Gaudreau, 2019). In Xu and Jaggars (2014, pp. 651-652) opinion, the online 
format showed a significantly negative relationship with standardized course grade, and performance gaps in 


https://doi.org/10.33225/jbse/20.19.1106 


1117 


—<LYYy 


Journal of Baltic Science Education, Vol. 19, No. 6A, 2020 


THE EFFECTS OF ONLINE TEACHING ON STUDENTS’ ACADEMIC PROGRESS IN STEM ISSN 1648-3898 
(PP. 1106-1124) 


/Print/ 


ISSN 2538-7 138 /oniines 


online courses were wider in some academic subject areas than others. In addition to the difference across course 
categories, Kelley and Knowles (2016, p.2) remarked a universal STEM integration language necessary for both 
research and practical implementation. 

Therefore, the influences of online educational format across STEM disciplines on student academic per- 
formance requirements were examined. Understanding the previous general knowledge gaps, this research 
proposed to examine the impact of online teaching challenges and concerns on students’ academic progress 
across the different STEM disciplines. The research has some strong arguments because of the sampling ap- 
proaches who represent the real-world population. Moreover, the participants were diverse in terms of teaching 
experience, school settings, gender and STEM subjects taught. So, the research used a sample of 1444 in-service 
teachers who teach STEM disciplines across Romania to respond to the four questions (Q,-Q,) associated with 
eighteen hypotheses regarding the effects of teachers’ pedagogical and technological perceived risks during 
only online classes on student's academic progress mediated by student engagement and school settings for 
each of the seven STEM disciplines. 

The first challenge of this research was the measurement of students’ engagement toward learning different 
STEM disciplines. Existing research studies for assessing students’ engagement, do not include explicit descrip- 
tions of engagement, a diversity of unexpressed aspects for engagement are not being considered (Trowler, 
2010). Consequently, this research has developed and validated a scale, which precisely measures three basic 
attributes of student behavioral engagement, including two items (i.e., following the rules, and active participa- 
tion) similar with the items proposed by Zyngier (2008), attention similar with the item proposed by M.-T. Wang 
et al. (2016). This research has an important contribution to the STEM education domain because it answers 
to some questions from researchers. For example, M.-T. Wang et al. (2016) proposed that the influences of the 
academic results by each element of engagement be measured. 

In the present research the strong association between the technological perceived risks and pedagogi- 
cal perceived risks caused by only online education was supported in all groups. Most of the previous research 
focused on topics such as: investigation of the perceived risks in users’ technology adoption (Im, Kim, & Han, 
2008), exploration of the teachers’ resistance to technology integration (Howard, 2013) or evaluation of the 
perceived risk of virtual learning community effects on user adoption behavior (Xie, 2017). 

In summary, the results of the present research indicated the following clear remarks: 

e the technological perceived risks are strongly positively correlated with pedagogical perceived risks 
for all STEM disciplines; 

e thetechnological perceived risks had a medium size effect on student engagement in case of math- 
ematics, informatics, biology and geography and a small effect in case of technologies, chemistry, 
and physics; 

e student engagement had a medium size effect on student academic progress in case of geography 
and a small effect in case of mathematics, technologies, informatics, biology, chemistry, and physics; 

e negative relationship between technological perceived risks and student academic progress differed 
in magnitude and significance across STEM disciplines; in case of biology the technological perceived 
risks had the highest negative influences on academic progress; also, in case of informatics, chem- 
istry, and geography the relationship between technological perceived risks and student academic 
progress were not statistically significant; 

A negative relationship between pedagogical perceived risks and student academic progress differed in 
magnitude and significance across STEM disciplines; in case of chemistry the pedagogical perceived risks had 
the highest negative influences on academic progress; also, in case of informatics, biology, and physics the 
relationship between pedagogical perceived risks and student academic progress is not statistically significant. 

The first hypothesis, which stated that technological challenges are strongly positively correlated with the 
pedagogical ones was supported in case of all STEM disciplines. This finding might be justified in the light of 
TPACK framework (i.e., technological knowledge (TK), pedagogical knowledge (PK), and content knowledge 
(CK)) (Tomte et al., 2015). This result appeared to be contradictory with the findings obtained by Tomte et al. 
(2015) who observed that within the educational process, teachers did not regard pedagogy and technology as 
blended elements, but instead as two different and independent ones. The second and third hypotheses that 
foreseen the correlation between pedagogical and technological risks and school settings were only partially 
validated, in several STEM disciplines. These results can only be explained by future measuring of the teacher's 
digital skills levels, and the degree of their participation in initial and continuing pedagogical training courses 
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across school settings (i.e., rural, medium-urban, and large-urban). As expected, the technological challenges 
significantly influence student’s engagement across all STEM disciplines as opposed to the pedagogical ones 
that do not influence engagement of students who participated in informatics, biology, and chemistry class 
(Table 4). With concern to the hypothesis H,, it seemed that school settings positively and significantly affected 
student academic progress only in case of teaching chemistry, probably due to the impossibility to carry out 
practical demonstrations. 

Importantly, findings of present research demonstrated that behavioral engagement has a negative sig- 
nificant contribution to student academic progress for all STEM disciplines. The outcomes expanded knowledge 
regarding the negative influences of challenges faced by teachers and concerns on academic progress and 
advised that technological risks are strongly positively correlated with the pedagogical ones for all STEM dis- 
ciplines. Other consequences resulted from the findings were the differences in terms of student engagement 
across STEM disciplines, under technological challenges influences. Moreover, student engagement significantly 
influences the student academic progress for all STEM disciplines, but the effects differed in magnitude. Biol- 
ogy discipline was found to be most affected by technological challenges and chemistry was found to be most 
affected by pedagogical challenges. 

Multigroup hypotheses (H,-H,,) were inter-correlated with direct hypotheses. All of these hypotheses were 
validated by present research, with the exception of the hypothesis H,, which postulated that negative relationship 
between technological perceived risks and student academic progress was similar for all students who studied 
STEM disciplines, and hypothesis H,, which postulated that negative relationship between pedagogical perceived 
risks and student academic progress was similar for all students who studied STEM disciplines. Academic progress 
of students might depend on the technological and pedagogical challenges, but only in some particular situation. 


Conclusions and Implications 


Currently, the measures that limit impact of the COVID-19 pandemic have disturbed the education systems 
from all over the world. Because the effects of this unprecedented and hazard situation on educational outputs 
are already visible and will have consequences on long-term, there is a clear need for improving online learn- 
ing and teaching strategies. In this context, new responsibilities for teachers were being created to integrated 
methods for K-12 STEM education using online practices experiences. 

More than ever, in secondary schools, expectations for online technology use are necessary to be supportive 
to students’ academic achievements. However, investigations regarding challenges in online STEM education 
met by in-service teachers were not extensively explored by researchers. This situation has significant nega- 
tive effects for effective integration of STEM education. Present research findings focused to identify the role 
and the effect of perceived risk of online instruction across STEM disciplines. Based on these data, the research 
was designed to identify a comprehensive model of teachers’ risk perceptions related to online education, as 
well as the effects of risk on academic progress across STEM disciplines. The multigroup analysis revealed the 
magnitude and significance of the links between technological correlated with pedagogical challenges, and 
academic progress across STEM disciplines. 

The current research has some limitations that initiate new directions of investigations in the future. The 
experimental data were collected in a special condition recognized by everyone, the Covid-19 pandemic emer- 
gency. Even if the sample was large, the responses of the research participants may not be reproducible, because 
they were under the strong influence of the current global health crisis. Nevertheless, this sample composed by 
STEM teachers has the advantage of homogeneity of groups which facilitates multivariate comparison of data. 
The other advantage of the sampling of in-service teachers was that they were very interested in the research 
subject, and they quickly answered to the online questionnaire. Future research studies will be designed to 
replicate the present one using more diverse samples in terms of student engagement dimensions, as well as 
strategies for online STEM education. 

All findings suggested that the student behavioral manifestations have a key mediated role between 
predictors and the output of research. If a teacher intends to online teach STEM subject, the student engage- 
ment factor must be taken into consideration. Reducing students’ behavioral problems and stimulating their 
engagement in online educational activities, the academic progress will increase, even if the technological and 
pedagogical challenges remain. Research findings will contribute to continual improvement of the quality of 
online STEM education. 
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Effects Size of Path Analyses 
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